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ABSTRACT 

The parameter estimation method that based on the minimum residual sum of squares is unsatisfactory in the 
presence of multi collinearity. In (1970) Hoerl and Kennard introduced an alternative estimation approach which is called 
the ridge regression (RR) estimator. In RR approach, ridge parameter plays an important role in the parameter estimation. 
Many researchers are suggested various methods for determining the ridge parameter for the RR approach and the 
ygeneralized their methods to be applicable for the logistic ridge regression (LRR) model. Schaeffer et al. (1984) was the 
first who proposed a LRR estimator. In this article, new methods for choosing the ridge parameter for logistic regression 
(LR) are proposed. The performance of the proposed methods are evaluated and compared with other models that having 
different previously suggested ridge parameter through a simulation study in terms of mean square error (MSE). 
The developed technique in this communication seems to be very reasonable because of having smaller MSE. The results 
from the simulation study generally show that all the LRR estimators have alower MSE than the maximum like lihood 
(ML) estimator and our suggested LRR estimators were superior in most of the cases. 

KEYWORDS: Logistic Regression, Maximum Likelihood, Monte Carlo Simulations, MSE, Multicollinearity, Ridge 
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INTRODUCTION 

The concept of multi collinearity was first introduced by Frisch (1934), which occurs when the independent 
variables in a multiple regression model are collinear. This problem, which isvery common in applied researches, causes 
high variance and instable parameter estimates when estimating both linear regression models using ordinary least 
squares(OLS) technique and the LR model using the maximum likelihood estimation (MLE) method. There are several 
ways to solve this problem. One popular way to deal with this problem is called the ridge regression that first proposed by 
[Hoerl and Kennard (1970)]. The RR is known as an efficient remedial measure for the linear regression model and the LR 
model. A lot of researches mainly focused on different ways of estimating the ridge parameter. The authors proved that 
there is a non-zero value of such ridge parameter for which the MSE for the coefficients p using the RR is smaller than 
theMSE of the OLS estimator or the ML estimator of the respective parameter. Many authors have worked with this area 
of research and developed and proposed different estimators for the RR parameter. To mention a few, Hoerl and Kennard 
(1970a), Hoerl et al. (1975),McDonald and Galarneau (1975), Lawless and Wang (1976), Schaeffer et al. (1984), 
Khalaf and Shukur (2005), Alkhamisi et al. (2006), and Muniz and Kibria (2009). 
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The main goal of this paper is to suggest some new methods for choosing the ridge parameter k for LR. 
The performance of these proposed methods is evaluated by comparing them with other previously suggested models that 
having different ridge parameter based on a simulation study in terms of MSE. Very promising results for our suggested 
methods are shown. 

METHODOLOGY 

In this section we propose some LRR estimators for estimating the ridge parameter k based on the work of Hoerl, 
Kennard and Baldwin in (1975), Schaefer et al. in (1948) and Dorugadein (2010). 

Model and Estimation 

Logistic regression is a widely used statistical method, the i th value ofthe vector of the response variableF nxl of 
the regression model is Bernoulli distributed with the following parameter value: 

e xi'p 

* i(Xi)= rr^ 

Where x { = [1, x i:L , x i2 , x i3 , ...x ip ] is the i th row of data matrix X nX ( p+1 ) which is a vector of p independent 
variables and constant,p = (p Q , p x p 2 ---.Pp) is (p+l)xlvector of the coefficients (unknown parameters), n is the sample 
size. 

The most common method of estimating P is to apply the maximum likelihood estimation (MLE) approach, the 
ML estimator of p is given by: 

^ I a=[X'WX]- 1 X'WZ l P (2) 

Where W is a square matrix of order n with elemental — 7ri),Z is an nxl column vector with the i th elements: 
Zi = logit^ifXi)] + rr—rr. The asymptotic covariance matrix of the ML estimator equals: 

711(1-710 

MPmle) = cov (Pmle) = (XWX)- 1 

= {X'diag ftd - Jtj)] X}" 1 (3) 

The MSE of the asymptotically unbiased P MLE is: 

mse = e(0 mus -p)XPmle-P) 

= Tr[Vartf MLE )]=r j=1 Y 

Aj (4) 

Where A,j is the j th eigenvalue of the X'WX matrix. One of the drawbacks of using the MLE approach is that the 
MSE of the estimator becomes inflated when the independent variables are highly correlated because some of the eigen 
values will be small. As a remedy to this problem, caused by the multicollinearity, Schaefer et al. (1984) proposed the 
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following LRR estimator. 



p k = {X'WX -f klpT^WX^v 



(5) 



The MSE of the LRR estimator equals: 



MSE=E(fl k -p)(0 k -p)n 



L, 0, + k)* ^ + k) 



(6) 



There are several researcher mainly focused on different ways of estimating the ridge parameter k [1] [13]. 
The Ridge Parameter 

Estimating the value of the ridge parameter k is an important problem in the RR method. Many different 
techniques for estimating k have been proposed by various researchers. The RR estimator does not provide a unique 
solution to the problem of multi collinearity but provides a family of solutions, because here is no specific rule for how to 
choose the ridge parameter. These solutions depend on the value of K which is the diagonal matrix of the non-negative 
constants kj. A useful procedure uses K = kl, k>0. However, several methods have been proposed for the linear RR model, 
and these methods have been generalized to be applicable for the LRR model. The most classical RR parameters are 
summarized in Table 1 . 



Tablel: Some Common RR Parameters 



Author 



Ridge parameter 



Proposed by Hoerl and Kennard (1970), 



kj=72 j = l,2,...p 
Pi 



Proposed by Hoerl and Kennard (1970) 



W = 



HK — 



Suggested byHoerl, Kennard and Baldwin (1975)[4] 



^hkb — 



pa 



pa 



gj SUA 2 



Proposed by Schaefer et al. (1984) 



ksRw — ~T 



Suggested by Khalaf and Shukur (2005) 



k ks = 



(n - p - l)a 2 + A, max [3 



Suggested by Dorugade and Kashid (2010) 



pa' 1 
k n = max 0 , — 7 r- 

aa nfviR) 



Wherea is the residual variance of the raw residuals divided by the degrees of freedoms (n - p -1), A, max is the 
largest eigenvalue of the matrix X X and VIFj = — ^-jis the variance inflation factor of the j th regressor. 

l-Rj 
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New Proposed Ridge Parameter 

In this section, three different methods of specifying the ridge parameter k will be proposed. Those three methods 
are considered to be a modification of three others ridge parameters proposed elsewhere. Our main goal is to give three 
new estimators with smaller MSE value compared with other previously suggested ridge estimators. The first new 
proposed ridge parameter, k SA1 , and hence its estimator is a modification of the estimator which is proposed by 
[Dorugade (2010)]. The mathematical formula of k SA1 is as follows: 

k SA1 = max ( 0,^7- -7-^: 1 \n 



Wherep is the ML estimator of p.By squaring the term 



n(VIF|) 



the value of ridge parameter will be 



increased, and as a consequence the bias of theproposed estimator will be also increased, and this will reduce the MSE 
ofthe corresponding ridge estimator. 

The second and the third modified ridge parameters are given by the following formulas: 



ksA2 — 



kc A 3 — 



p(VIFj) r 



(8) 



L SA3 — „2 



p(VIFj) r 



(9) 



The k SA2 ridge parameter is an enhancement of the ridge parameter which is given by [Hoerl, Kennard and 
Baldwin] in (1975). While k SA3 is a modification of the ridge parameter which is suggested by [Schaefer et al.] in (1948). 



Our goal is to multiply those two previously suggested ridge parameters by the term 



which is often 



LptVIFj)^ 

greater than one. So the value of the bias of the two new suggested estimators will be increased, and this will give an 
opportunity for a large reduction of the MSE criterion of the two new suggested ridge estimators. 

Simulation Study 

In this section, the performance of the three suggested ridge estimators is evaluated over several different ridge 
estimators. Since a theoretical comparison is not possible, a simulation study is conducted in this section. The design of a 
good simulation study is depended on: 

• What factors are expected to affect the properties of the estimators under investigation, and 

• What criteria are being used to judge the results. 
Factors Affecting the Properties of Estimators 

In this section, a brief description of the selected factors that is used in the simulation study with different values 
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will be presented. 

• The Strength of Correlation among the Predictor Variables (p 2 ) 

The most obvious factor that affects the properties of the different estimators is the degree of correlation between 
the independent variables. The four different degrees of correlation that are used in this simulation study are: 

p 2 = 0.70, 0.80, 0.90 and 0.95. 

• The Number of Independent Variables(P) 

Another factor that has an obvious effect on the evaluation of the estimators is the number of independent 
variables. The main interest of varying this factor is to see which ridge parameter is the best for specific number of 
independent variables. In most simulation studies the proposed ridge estimator is calculated using a fairly low number of 
predictor variables (2 and 4 is the most common selected value of p)[14]. Hence, there is a need to conduct an investigation 
where more variables are considered to see the effect of increasing the number of independent variables on the 
performance of the ridge estimators. The number of independent variables that is used in the simulated models is equal to 
2,3,4,5, 10. 

• The Sample Size (n) 

Another consideration that is taken into account is the sample size n.Actually, when comparing different 
estimation methods, increasing the n is supposed to have a positive effect on the MSE, as increasing the n leads to a lower 
variance of the estimated parameters. Therefore, it is interesting to investigate the gain of using LRR when n is both small 
and large. The sample size is increased with the number of independent variables (p). Many papers show that to obtain 
meaningful results from the LR model, the sample size is needed to be adjusted. Therefore, the number of observations that 
are used in this simulation study is depend on 20p+10, 30p, 40p, 60p, and lOOp, respectively [13] [17]. 

Criteria for Measuring the Goodness of an Estimator 

The MSE is used as a criterion to measure the goodness of the estimator. It is used to compare the new three 
proposed ridge estimators with other four previously suggested ridge estimators together with the ML estimator. 
For a given values of p, n, and p 2 the set of predictor variables are generated. Then the experiment was repeated 1,000 
times by generating new error terms. After that the values of the ML estimator, also the previously suggested and the 
modified ridge parameters k and their corresponding ridge estimators as well as the average MSE (AMSEs) are evaluated 
for each estimator. 

Generation of Independent and Dependent Variables 

Following Gibbons (1981), and to achieve different degrees of collinearity, the predictor variables are generated 
using the following equation: 



i = 1,2, ...,n, ] = 1,2, ... ,p, p 2 represents the correlation between any two predictor variables and z^are 
independent standard normal pseudo-random numbers. The n observations for the dependent variable are obtained from the 



Xij = (l-p^Zij+pzt 



where ^ 



(10) 
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Bernoulli (ttj) distribution in Equation (1). The values of the parameter s/$ lf p 2 , ...,/? p are chosen so that/?! = /? 2 = ••• = 
/? p and2y =1 /?; = 1, which is common restrictions in many simulation studies; [12] .The value of the intercept is another 
important factor since it equals the average value of the log odds ratio. Hence, when the intercept equals zero then there is 
an equal average probability of obtaining one and zero. While, when the intercept is positive then the average value of the 
log odds ratio is positive which means that there is a greater probability of obtaining one than zero. Finally, when the value 
of the intercept is negative the opposite situation occurs which means that there is a greater probability of obtaining zero 
than one. Accordingly, the value of the intercept in the simulation study is chosen to be zero. [13] 

RESULTS AND DISCUSSIONS 

In this section, the main results of the Monte Carlo simulation concerning the properties of the estimation method 
for choosing the ridge parameter have been presented. The results of the simulated AMSEs are summarized in Tables [2-6] 
and Figures [1-10]. Those Tables and Figures show the effects of changing the sample sizes and the correlation coefficient 
values between the independent variables on the performance of ML and different ridge estimators. 

According to our simulation study many conclusion can be drawn on the performance of the ML and different 
ridge modified and previously suggested estimators, these conclusion, can be summarized as follows: 

• Almost all the cases indicates that the ML estimator performs worse than the modified and previously suggested 
ridge estimators except when (n=200 and p=0.7) the performance of estimator based on k SA3 was not good. 

• Our first modified ridge estimator based on k SA1 perform better than k D estimator in all cases. 

• The second suggested ridge estimator based on k SA2 as a modification of k HKB is also performs better than 
estimator based on kj]KB in most cases. 

• Also k SA3 gives much better prediction results comparable with the ML estimator and the other modified and 
previously suggested estimators, this estimator seems to be superior at most of the cases. 

• The estimator based onk SA2 is better than the estimator based on/c 5i41 when the correlation is not too high, but with 
the strong correlation the estimator based onk SA1 become sbetter than the estimator based onk SA2 . With increasing 
the sample size, the estimators based on k SA1 , k SA2 zre approaching to each other and the difference between them 
becomes small. The generally, estimator based onk SA2 is best in most cases. 

• The ridge estimators that based on the parameters k SA1 and k HKB , have approximately the same results in most of 
the cases. The reason behind that is the k SA1 is a modification of the k D parameter which, in the origin, 

is a modification of the k HKB parameter. More specifically, when squaring the term 



that is included 



|n(VIFj) r 

wk HKB i\s value approaches to zero and the value of /c 5i41 and the value of k HKB become the same. 
CONCLUSIONS 

The performance of the three new proposed ridge estimators based onk SA1 , k SA2 zn& k SA3 are shown to be better 
than the ML estimator in most of the cases. Our three suggested modifications give better prediction results than the 
previously suggested ridge estimators in most of the cases. Our third suggested ridge estimator that based on the ridge 
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estimator which is proposed by [Schaefer et al. (1984)] looks superior to all the studied ML and ridge estimators as it has 
smaller AMSE in most of the cases. 
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APPENDICES 



Table 2: The AMSE of the ML and Different Ridge Estimators, 
For P=2 and Different Correlation and Sample Size 



P 2 


N 


MLE 


^HK 


^HKB 


ksBW 




ksAi 


ksA2 


1*SA3 


0.7 


50 


0.8181 


0.7133 


0.6690 
8 


0.5540 
6 


0.6733 


0.6691 
1 


0.5642 


0.5361 


0.80 


1.2106 


1.0039 


0.9188 


0.6970 


0.9248 


0.9189 


0.7756 


0.6650 


0.90 


2.4223 


1.8110 


1.5835 


1.1182 


1.5942 


1.5835 


1.4031 


1.0513 


0.95 


4.9138 


3.2621 


2.7705 


1.8680 


2.7898 


2.7706 


2.7045 


1.8408 


0.7 


60 


0.6619 


0.5856 


0.5543 


0.4462 


0.5566 


0.5543 


0.4656 


0.4617 


0.80 


0.9819 


0.8291 


0.7661 


0.5854 


0.7694 


0.7661 


0.6395 


0.5712 


0.90 


1.9635 


1.4983 


1.3246 


0.9386 


1.3305 


1.3247 


1.1467 


0.8808 


0.95 


3.96*1 


2.6887 


2.2783 


1.5345 


2.2886 


2.2783 


2.1565 


1.4877 


0.7 


80 


0.4648 


0.4195 


0.4014 


0.3321 


0.4022 


0.4014 


0.3374 


0.3687 


0.80 


0.6921 


0.5980 


0.5603 


0.4399 


0.5615 


0.5603 


0.4622 


0.4499 


0.90 


1.3S63 


1.0880 


0.9-1- 


0.6997 


0.973S 


0.9717 


0.S0S4 


0.6601 


0.95 


2.8220 


1.9672 


1.6829 


1.1460 


1.6867 


1.6829 


1.5109 


1.0870 


0.7 


120 


0.2925 


0.2712 


0.2629 


0.2259 


0.2631 


0.2629 


0.2251 


0.2776 


0.80 


0.4369 


0.3922 


0.3736 


0.3061 


0.3739 


0.3736 


0.3098 


0.3379 


0.90 


0.8767 


0.7257 


0.6627 


0.4985 


0.6633 


0.6627 


0.5378 


0.4888 


0.95 


1.7867 


1.3215 


1.1505 


0.8061 


1.1516 


1.1505 


0.9733 


0.7558 


0.7 


200 


0.1695 


0.1613 


0.1584 


0.1419 


0.1584 


0.1584 


0.1402 


0.2032 


0.80 


0.2526 


0.2344 


0.2275 


0.1951 


0.2276 


0.2275 


0.1930 


0.2447 


0.90 


0.5086 


0.4439 


0.4172 


0.3282 


0.41-4 


0.4172 


0.3354 


0.3453 


0.95 


1.0359 


0.8213 


0.7359 


0.5318 


0.7361 


0.7359 


0.5888 


0.5045 



Table 3: The AMSE of the ML and Different Ridge Estimators, 
for P=3 and Different Correlation and Sample Size 
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in 



P 2 


N 


MLE 




^HKB 


ksBW 


k n 


ksAi 


ksA2 


kg A3 


0.7 


70 


1.1095 


0.9S62 


0.9006 


0.7162 


0.9025 


0.9006 


0.S293 


0.66S3 


O.SO 


1.7151 


1.4736 


1.3073 


1.0024 


1.3102 


1.3073 


1.2257 


0.9461 


0.90 


3.5942 


2.8829 


2.4511 


1.7856 


2.4564 


2.4511 


2.4193 


1.7635 


0.95 


7.4426 


5.5414 


4.5896 


3.2725 


4.5995 


4.5896 


4.8412 


3.4348 


0.7 


90 


0.S163 


0.7342 


0.6779 


0.5501 


0.6787 


0.6779 


0.6191 


0.5159 


O.SO 


1.2672 


1.0994 


0.9866 


0.7674 


0.9878 


0.9866 


0.9112 


0.7201 


0.90 


2.6577 


2.1570 


1.8448 


1.3644 


1.8470 


1.8448 


1.7810 


1.3239 


0.95 


5.5317 


4.1948 


3.4649 


2.5019 


3.4689 


3.4649 


3.5693 


2.5660 


0.7 


120 


0.5873 


0.5381 


0.5043 


0.4176 


0.5047 


0.5043 


0.4587 


0.3936 


O.SO 


0.9109 


0.8105 


0.7392 


0.5873 


0.7397 


0.7392 


0.6752 


0.5476 


0.90 


1.9105 


1.5893 


1.3799 


1.0285 


1.3807 


1.3799 


1.3012 


0.9798 


0.95 


3.9704 


3.0618 


2.5431 


1.8301 


2.5447 


2.5431 


2.5448 


1.8315 


0.7 


180 


0.3743 


0.3499 


0.3345 


0.2863 


0.3346 


0.3345 


0.3048 


0.2766 


O.SO 


0.5796 


0.5281 


0.4934 


0.4044 


0.4936 


0.4934 


0.4473 


0.3797 


0.90 


1.2155 


1.0450 


0.9293 


0.710S 


0.9295 


0.9293 


0.8539 


0.6638 


0.95 


2.5322 


2,0292 


1.7111 


1.2469 


1.7115 


1.7111 


1.6468 


1.2074 


0.7 


300 


0.2175 


0.2075 


0.2019 


0.1798 


0.2019 


0.2019 


0.1859 


0.1S1S 


O.SO 


0.3366 


0.3147 


0.3010 


0.2573 


0.3010 


0.3010 


0.2735 


0.2493 


0.90 


0.7057 


0.6299 


0.5774 


0.4600 


0.5775 


0.5774 


0.5210 


0.4286 


0.95 


1.4681] 


1.2285 


1.0666 


0.S016 


1.0663 


1.0662 


0.9853 


0.7534 



Table 4: The AMSE of the ML and Different Ridge Estimators, 
For P=4 and Different Correlation and Sample Size 



P 2 


N 


MLE 




k HKB 


k SBW 






k SA2 


k SA3 


0.7 


90 


1.3154 


1.1856 


1.0662 


0.8625 


1.0672 


1.0668 


1,0217 


0.8240 


O.SO 


2.0812 


1.8253 


1.5951 


1.2526 


1.5967 


1.5951 


1.5556 


1.2199 


0.90 


4.4369 


3.6987 


3.1023 


2.3505 


3.1053 


3.1023 


3,1544 


2,3897 


0.95 


9.3517 


7.4423 


6.1193 


4.5684 


6.1253 


6.1193 


6,5239 


4.8582 


0.7 


120 


0.9293 


0.S485 


0.7743 


0.6352 


0.7747 


0.7743 


0,7356 


0.6012 


O.SO 


1.4643 


1.3010 


1.1506 


0.9103 


1.1512 


1.1506 


1,1069 


0.S728 


0.90 


3.1266 


2.6393 


2.2224 


1.6894 


2.2235 


2.2224 


2.2184 


1.6858 


0.95 


6.5701 


5.2548 


4.2960 


3.1915 


4.2982 


4.2960 


4.4993 


3.3379 


0.7 


160 


0.6704 


0.6215 


0.5769 


0.4839 


0.5771 


0.5769 


0,5460 


0.4567 


O.SO 


1.0555 


0.9536 


0.S600 


0.6942 


0.8602 


0.S600 


0.S196 


0.6594 


0.90 


2.2559 


1.9320 


1.6551 


1.2715 


1.6556 


1.6551 


1,6241 


1.2470 


0.95 


4.7336 


3.8324 


3.1498 


2.3590 


3,1507 


3.1498 


3.2339 


2.4190 


0.7 


240 


0.4294 


0.4040 


0.3825 


0.3298 


0.3825 


0.3825 


0,3613 


0.3131 


O.SO 


0.6763 


0.6216 


0.5736 


0.4748 


0.5737 


0.5736 


0,5419 


0.4492 


0.90 


1.445 


1.2695 


1.1079 


0.3691 


1.10S0 


1.1079 


1,0649 


0.S348 


0.95 


3.0323 


2.507S 


2.0796 


1.5707 


2.0798 


2.0796 


2,0769 


1.5685 


0.7 


400 


0.2495 


0.2387 


0.2306 


0.2051 


0.2306 


0.2306 


0,2187 


0.1978 


O.SO 


0.3930 


0.3692 


0.3495 


0.2994 


0.3495 


0.3495 


0,3295 


0.2844 


0.90 


0.8388 


0.7559 


0.6831 


0.5498 


0.6831 


0.6831 


0,6457 


0.5210 


0.95 


1.7644 


1.5117 


1.2902 


0.9962 


1.2903 


1.2902 


1.2501 


0.9669 



Table 5: The AMSE of the ML and Different Ridge Estimators, 
For P=5 and Different Correlation and Sample Size 
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P 2 


N 


ML.E 




^HKB 


^SBW 


k D 




ksA2 


ksA3 


0.7 


110 


1 ,4,5 S3 


1.3227 


1.1736 


0,9564 


1 . 1 742 


1 ,1736 


1 ,1451 


0.9287 


O.SO 


2.3299 


2.0606 


1.7771 


1.4075 


1.7781 


1 ,7771 


1 ,7624 


1.3940 


0.90 


5.0403 


4.2767 


3.5383 


2.7321 


3.5402 


3,5383 


3,6349 


2.8094 


0.95 


10.6464 


8.6723 


7.0239 


5,3775 


7.0277 


7,0239 


7,4950 


5.7297 


0.7 


150 


1.0081 


0.9274 


0.S395 


0,6981 


0.S398 


0.S395 


0.8130 


0.6720 


O.SO 


1 .6055 


1 .4421 


1 .2652 


1 ,0200 


1 .2655 


1 .2652 


1.2398 


0.9965 


0.90 


3.4581 


2.9697 


2.4812 


1,9340 


2.4818 


2.4812 


2,5080 


1.9564 


0.95 


7.3200 


5.9994 


4.8627 


3,7198 


4.8639 


4,8627 


5,1057 


3.9057 


0.7 


200 


0.7365 


0.6861 


0.6319 


0,5327 


0.6320 


0,6319 


0,6094 


0.5102 


O.SO 


1 .1745 


1.0694 


0.9559 


0,7777 


0.9561 


0,9559 


0,9293 


0.7524 


0.90 


2.5362 


2.2140 


1.8773 


1,4710 


1.8776 


1,8773 


1,8731 


1.4670 


0.95 


5 ,3682 


4.4652 


3.6522 


2,7984 


3.6527 


3,6522 


3,7776 


2.8930 


a ~f 

0. / 


300 


A A HF 1 3 

0.4713 


0.4443 


0.41o>0 


0.3595 


0.41&0 


A J 1 OA 

0,41 SO 


A 4 A 1 f\ 


A ■ > A 3 i~\ 

0.3439 


O.SO 


0.7515 


0.6943 


0.6350 


0.5273 


0.6350 


0,6350 


0,6122 


0.5057 


0.90 


1.6239 


1.4385 


1.2463 


0,9894 


1.2464 


1.2463 


1.2230 


0.9696 


0.95 


3.4379 


2.9092 


2.4019 


1 S656 


2,4021 


2.4019 


2.4330 


1.8892 


0.7 


500 


0.2758 


0.2644 


0.2542 


0,2259 


0.2542 


0,2542 


0,2449 


0.2170 


O.SO 


0.4384 


0.4133 


0.3884 


0,3328 


0.3884 


0,3884 


0,3731 


0.3178 


0.90 


0.9448 


0.S560 


0.7641 


0,6188 


0.7641 


0,7641 


0,7382 


0.5959 


0.95 


1,9990 


1.7278 


1.4597 


1,1425 


1.4597 


1.4597 


1.4432 


1.1295 



Table 6: The AMSE of the ML and Different Ridge Estimators, 
For P=10 and Different Correlation and Sample Size 



P 2 


N 


MLE 








k n 


k S Al 


k S A2 


k S A3 


0.7 


210 


2,0428 


1.8974 


1.6529 


1 .3933 


1.6531 


1.6529 


1,6555 


1.3960 


O.SO 


3.3623 


3.0749 


2.6098 


2.1720 


2.6100 


2,6098 


2,6382 


2.1994 


0,90 


7.4938 


6.6820 


5,4687 


4.5201 


5.4692 


5.4687 


5,6212 


4.6534 


0,95 


16.1358 


14.0717 


11.2645 


9,3006 


11.2656 


11.2645 


11,7697 


9.7128 


0,7 


300 


1.3410 


1.2554 


1.1131 


0,9457 


1.1131 


1.1131 


1.10S7 


0.94091 


O.SO 


2.1956 


2.0215 


1,7356 


1.4561 


1.7356 


1.7356 


1,7428 


1.4632 


0.90 


4.8666 


4.3506 


3,5743 


2,9680 


3.5744 


3.5743 


3,6473 


3.0317 


0.95 


10.4399 


9.0S65 


7.2557 


6,0123 


7.2560 


7,2557 


7,5294 


6.2332 


0,7 


400 


0.9733 


0.9174 


0,8258 


0,7059 


0.8258 


0.8258 


0.S199 


0.6988 


O.SO 


1.5917 


1.4744 


1.2849 


1,0782 


1.2849 


1.2849 


1.2839 


1.0772 


0.90 


3.5286 


3.1717 


2,6297 


2,1842 


2,6298 


2,6297 


2,6673 


2.2171 


0.95 


7.5740 


6.6173 


5,3002 


4,3918 


5.3003 


5,3002 


5,4662 


4.5264 


0.7 


600 


0.6248 


0.5934 


0,5461 


0,4695 


0.5461 


0,5461 


0,5400 


0.4621 


O.SO 


1.0220 


0.9543 


0,8485 


0,7122 


0.8485 


0.S4S5 


0.8427 


0.7061 


0,90 


2.2629 


2.0493 


1,7238 


1,4233 


1.7238 


1,7238 


1,7343 


1.4328 


0,95 


4.8502 


4.2590 


3,4317 


2,8279 


3.4318 


3,4317 


3.5087 


2,8910 


0,7 


1000 


0.3667 


0.3530 


0,3339 


0,2945 


0.3339 


0,3339 


0.3297 


0.2890 


O.SO 


0.5988 


0.5683 


0,5219 


0,4482 


0.5219 


0,5219 


0,5161 


0.4414 


0.90 


1.3243 


1.2182 


1,0551 


0,8761 


1.0551 


1,0551 


1,0518 


0.8730 


0.95 


2.8409 


2.5332 


2,0828 


1,7203 


2.0828 


2,0828 


2,1055 


1.7394 
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Figure 2: The AMSE of the ML and Different Ridge Estimators, 
For p=2, n=50, 80, 120 and 200 with Different Correlation 






Figure 4: The AMSE of the ML and Different Ridge Estimators, 
For P=3, N=70, 120, 180 and 300 with Different Correlation 
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Figure 6: The AMSE of the ML and Different Ridge Estimators, for 
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Figure 9: The AMSE of the ML and Different Ridge Estimators, for 
P=10, P=0.70, 0.80, 0.90 and 0.95 with Different Sample Size 
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Figure 10: The AMSE of the ML and Different Ridge Estimators, for 
P=10, N=210, 400, 600 and 1000 with Different Correlation 
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